1. Field of the Invention
The present invention relates to a camera calibration and rectification method and apparatus for a stereo camera, and more particularly, to a camera calibration and rectification method and apparatus for a stereo camera, which is capable of performing calibration and rectification at high speed.
2. Description of the Related Art
In general, the largest factor which causes a person to feel a three-dimensional (3D) effect is an effect of a spatial difference between left and right retinas, which occurs when the left and right eyes of the person see one object in different directions. A stereo camera using such an effect acquires different images through left and right lenses thereof, and displays a 3D image.
FIG. 1 shows a photograph of a stereo camera.
The stereo camera is implemented by using two left and right lenses and hardware for calculating 3D information. When a person sees an object through the eyes, the brain analyzes the object in 3D manners. In this ways, the stereo camera acquires left and right images of a space through the left and right lenses, and calculates 3D information of an object from the left and right images.
The stereo camera requires calibration and rectification to represent a 3D space.
Referring to FIG. 2, the calibration is to calibrate a distortion of the lenses and a tilt of an image sensor in the stereo camera. Referring to FIG. 3, the rectification is to precisely adjust a horizontal arrangement of the left and right lenses of the stereo camera.
The calibration is performed by the following process: a panel having cross stripes with a constant size is positioned at an arbitrary distance from the camera, and the camera acquires a plurality of pattern images based on various postures as shown in FIG. 2, and performs a calculation to extract calibration parameters from the acquired image information.
A camera calibration method which is currently used most frequently has been proposed by Zhengyou Zhang (refer to “A Flexible New Technique for Camera Calibration” (IEEE Trans, PAMI, Vol. 22, No. 11, November, 2000, pp. 1330-1334)).
Furthermore, the rectification is to acquire a rectification parameter using the calibration parameters of the left and right cameras.
FIG. 5 illustrate an imaged plane before and after rectification.
A left side of the FIG. 5 illustrates an imaged plane before rectification, and A right side of the FIG. 5 illustrates an imaged plane after rectification.
The rectification is performed to satisfy a condition where the optical axes of the left and right cameras of the stereo camera are parallel to each other as shown in FIG. 5, and to satisfy an epipolar constraint. A point W in the space is projected onto points M1 and M2 of left and right images. At this time, a plane represented by the points W, M1, and M2 is referred to as an epipolar plane, and a line segment at which the epipolar plane and an image plane meet each other is referred to as an epipolar line. The epipolar constraint is where a correspondence point to one point of a left side of the FIG. 5 exists on the epipolar line of a right side of the FIG. 5, and is a very important condition for the stereo camera.
A rectification method which is used most frequently has been proposed by Fusiello (refer to “A compact algorithm for rectification of stereo pairs” (Machine Vision and Applications, pp. 16-22, Vol. 12, 200.) by A. Fusiello, E. Trucco, A. Verri).
Referring to FIG. 3, before the rectification, a correspondence point to one point of the left image of FIG. 3 does not appear on an epipolar line (red horizontal line) of the right image. After the rectification, however, a correspondence point to one point of the left picture exists on the epipolar line of the right image.
FIG. 4 shows a panel disposed in various postures to acquire pattern images.
Referring to FIG. 4, the pattern images are acquired by tilting the panel having predetermined patterns displayed thereon in various directions in front of the camera, and the patterns may include check patterns with a constant distance therebetween, like a chessboard. During this process, the panel may be moved to acquire an image, and then postured in different positions and directions to acquire images.
Such a process has been manually performed, and dozens of minutes are required for acquiring the final calibration parameters.
When a stereo camera is manufactured, the camera calibration and rectification process requires the largest amount of time, and may become the largest factor which reduces productivity in mass production.
FIG. 6 is a flow chart showing a conventional calibration and rectification method for a stereo camera.
At step S610, a panel having grid patterns displayed thereon is prepared.
At step S620, a plurality of images of the panel are manually acquired from the left and right cameras as shown in FIG. 4.
At step S630, calibration for the left and right cameras is performed to acquire calibration parameters.
At step S640, a rectification parameter is acquired from the calibration parameters of the left and right cameras.
At step S650, the calibration and rectification parameters are applied to the stereo camera.
At step S660, an image to which the calibration and rectification parameters are applied is checked.
In the conventional calibration and rectification method, the pattern images in various postures are acquired by manually tilting the panel at an arbitrary angle in front of the camera at the step S620, and the calibration and rectification parameters are then calculated. Accordingly, since the manual operation requires a long time, the time required for the calibration and rectification of the stereo camera inevitably increases.